Many of today's digital video devices capture a color image using a color filter array (CFA) and an array of optical sensors that is overlaid by the CFA. The CFA has a plurality of filter elements, each of which overlays one of the optical sensors in the optical sensor array, and each filter element is designed to pass only one color of light. The most common type of CFA is the Bayer pattern filter (also referred to as a Bayer filter), an example of which is shown in FIG. 1. The Bayer filter comprises filter elements that pass three different colors of light: green (G), blue (B), and red (R). These filter elements are arranged in alternating rows of green-blue and red-green. With this pattern, red and blue filter elements appear on every other row and every other column, while green filter elements appear on every row and every column. This pattern exploits the human eye's higher sensitivity to green light.
As noted, each filter element passes light of only one color. Thus, in the Bayer filter of FIG. 1, filter element G1 passes only green light, filter element B2 passes only blue light, and filter element R9 passes only red light. This means that the optical sensor behind filter element G1 senses only the green portion of the light that hits filter element G1, the optical sensor behind filter element B2 senses only the blue portion of the light that hits filter element B2, and the optical sensor behind filter element R9 senses only the red portion of the light that hits filter element R9. Thus, because of the CFA, the image captured by the sensor array is a mosaic image. As used herein, the term mosaic image refers to an image in which there is only partial color information for each pixel of the image. In the case of a mosaic image captured using a Bayer filter, there is only information for one of the three color channels of green, blue, and red for each pixel.
After capturing a mosaic image through the Bayer filter, the array of optical sensors outputs a plurality of values. Each value comes from one of the optical sensors, and each value corresponds to a particular pixel of the image (each optical sensor represents a pixel). Since each pixel corresponds to one of the optical sensors, and since each optical sensor corresponds to one of the filter elements of the Bayer filter, each pixel in turn corresponds to one of the filter elements of the Bayer filter. For each pixel, only one of the three color values is provided by the optical sensor array. This color value will be referred to herein as the sensed pixel value. In the example of FIG. 1, the sensed pixel value provided by the optical sensor array for pixel 1 is for the green color channel. Thus, pixel 1 is referred to as a green pixel. Similarly, the sensed pixel value for pixel 2 is for the blue color channel; thus, pixel 2 is referred to as a blue pixel. Similarly, pixel 9 is referred to as a red pixel.
In order to render a pixel in full color, a color value is needed for all three of the color channels. Since only one sensed pixel value (corresponding to one of the three color channels) is provided by the sensor array for each pixel, the other two color values need to be derived for each pixel, which means that for each pixel, there is one sensed pixel value and two derived pixel values (corresponding to the other two color channels). A demosaicing process is performed on the mosaic image to calculate the two derived pixel values for each pixel. Through the mosaicing and demosaicing process, a color image can be captured and reproduced.
The sensed pixel values provided by the sensor array may and most likely will include some noise components. These noise components may be due to various factors, such as environmental conditions, manufacturing variations in the optical sensors, etc., and the amount of noise may vary from sensor to sensor. This noise in the sensed pixel values can lead to the introduction of unwanted, spurious colors in the demosaiced image. For example, if the noise in the sensed pixel value for a blue pixel causes the blue component of the pixel to be too high, it may cause the final demosaiced version of that pixel to take on a color that does not reflect the true color of that pixel in the original image. This can lead to a strange and unpleasant image. To prevent this, most image processing systems perform some form of de-noising.
De-noising is often done by averaging the sensed pixel values of nearby pixels on a color channel by color channel basis. For example, a blue pixel of interest may be denoised by taking the sensed pixel values of neighboring blue pixels and averaging them (according to certain weights given to each pixel) with the sensed pixel value of the blue pixel of interest to derive a de-noised pixel value for the blue pixel of interest. A green pixel may be de-noised in the same way by taking into account the sensed pixel values of neighboring green pixels, and a red pixel may be de-noised by taking into account the sensed pixel values of neighboring red pixels. Thus, pixels are typically de-noised on a color channel by color channel basis (in other words, pixels of different colors are de-noised separately based on neighboring pixels of the same color).
It has been observed by Applicants, however, that each color channel is often noisy in its own unique way. For example, the blue color channel may be noisy in a different way from the red color channel. It has been observed that if the various color channels are de-noised separately, the color channels may becoming exaggeratedly decoupled from each other, which may lead to the appearance of prominent and colorful noise in the demosaiced image, particularly in gray areas. This colorful noise (also referred to as chromatic noise) in the demosaiced image may render the image unpleasant. To present such an undesirable result from arising, an improved de-noising method is needed.